DEFINING RELATIONS OF INVARIANTS 
OF TWO 3x3 MATRICES 
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Abstract. Over a field of characteristic 0, the algebra of invariants of several 
n xn matrices under simultaneous conjugation by GL„ is generated by traces 
of products of generic matrices. Teranishi, 1986, found a minimal system of 
eleven generators of the algebra of invariants of two 3x3 matrices. Nakamoto, 
2002, obtained an explicit, but very complicated, defining relation for a similar 
system of generators over Z. In this paper we have found another natural set 
of eleven generators of this algebra of invariants over a field of characteristic 
and have given the defining relation with respect to this set. Our defining 
relation is much simpler than that of Nakamoto. The proof is based on easy 
computer calculations with standard functions of Maple but the explicit form 
of the relation has been found with methods of representation theory of general 
linear groups. 



Introduction 

Let K be any field of characteristic and let Xi = (xpq^, p,q ~ l,...,n, 

i = 1, . . . , d, he d generic n x n matrices. The conjugation of Xi with the invertible 
n X n matrix g, 

X, = {x(;^) ^ gX,g-' ^ {y^;j) , 

defines an action of the general linear group GL„ = GLn{K) on the polynomial 
algebra in n^d variables 

n^d = K[x^p^ \p,q = l,...,n, i = l,...,d] 

hy g * Xpq = Hpq, g G GL„. We denote by Cnd the algebra of invariants ■ 
Its elements are called the invariants under the action of GL„ by simultaneous 
conjugation of d matrices of size n x n. 

Traditionally, we call a result giving the explicit generators of the algebra of 
invariants of a linear group G a first fundamental theorem of the invariant theory of 
G and a result describing the relations between the generators a second fundamental 
theorem. 

For a background on the algebra of matrix invariants see e.g., |12[I5]. for the his- 
tory see It is known that Gnd is generated by traces of products of generic ma- 
trices tT:{Xi^ ■ ■ ■ Xi^ ) . For a fixed n an upper bound for the degree k is given in terms 
of Pl-algebras. By the Nagata-Higman theorem the nil algebras of bounded index 
are nilpotent, i.e., the polynomial identity x" = implies the identity xi ■ ■ ■ Xm = 0. 
Then k < m and for d sufficiently large this bound is sharp. A description of the 
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defining relations of Cnd is given in the theory of Razmyslov ^H] and Procesi [T^ 
in the language of ideals of the group algebras of symmetric groups. 

Explicit minimal sets of generators of Cnd arc known for n = 2 and any d and 
for n = 3 and d — 2. By a theorem of Sibirskii 21 , C2d is generated by tr{Xi), 
1 <i <d, tT{X^Xj), I <i <j <d, tT{X,XjXk), 1 < i < j < k < d. There are no 
relations between the five generators of the algebra C22, i-e., C22 — K[zi, . . . ,25]. 
For d = 3, Sibirskii 21 found one relation and Formanek ^] proved that all 
relations follow from it. Aslaksen, Tan and Zhu 4 found, mostly by computer, 
some defining relations for bigger d, as well as generators and some defining relations 
for the 2x2 matrix invariants for other classical groups. The centre of GL2 acts 
trivially on fl2d and we have also a natural action of PSL2 on U,2d- Since PSL2{C) 
is isomorphic to 503(C), we may apply invariant theory of orthogonal groups, see 
Procesi ^H] and Le Bruyn ^]. In particular, Drensky _7_ translated the defining 
relations of the invariants of SO3 and obtained the defining relations of C2d for all 
d. 

Teranishi |22] found the following system of generators of C32: 
tr(X), tr(r), tr(X2), tr(Xr), tr(r2)^ 

(1) 

tr(X3), tr(X2y), tT{XY^), tr(y3)^ tT{X^Y^),tTiX^Y^XY), 

where X, Y are generic 3x3 matrices. He showed that the first ten of these gen- 
erators form a homogeneous system of parameters of C32 and C32 is a free module 
with generators 1 and ti{X'^Y'^XY) over the polynomial algebra on these ten el- 
ements. (Teranishi found also a set of generators and a homogeneous system 
of parameters of C42 ) Abeasis and Pittaluga PP found a system of generators of 
Csd in terms of representation theory of the symmetric and general linear groups, 
in the spirit of its use in theory of Pl-algebras. 

The algebras Cnd have a natural multigrading which takes into account the 
degrees of the traces tr{Xi^ ■ ■ ■ Xi^) with respect to each of the generic matrices 
Xi, . . . , Xd- The Hilbert series of Cnd is defined as the formal power series 

H{Cnd. ii, . . . , t,) = ^ dim{cl!'r-''''^)t ■■■t'T 

ki>0 

with coefficients equal to the dimensions of the homogeneous components (j^^^''"''^'''^ 
of degree (fci, . . . , kd). It carries a lot of information for the algebra. The Hilbert 
series of C2d are calculated e.g., in J^l and those of C32 and C42 in |22|. Van 
den Bergh reduced the determination of the Hilbert series of Cnd to a problem 
about flows in a certain graph and obtained an important consequence for the 
denominators of the rational functions in the explicit form of the series. He also 
evaluated H{Cnd, ti, . ■ . ,td) for several small n and d. 

We shall need the Hilbert series of C32 obtained by Teranishi [221 as a conse- 
quence of his description of C32 : 

1 + t^t^ 

(2) H{Cs2,h,t2) - ^i_t^)(l-t2)q2{ti,tMtiMil-m)' 

where the commuting variables ti and t2 count, respectively, the degrees of X and 

Y and 

92(^1,^2) = (l-t?)(l-tl<2)(l-i2), 

q3{ti,t2) = {l-tl){l-tlt2){l-titl){l-tl). 
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It follows from the description of the generators of C32, that tr{X^Y^XY) satisfies 
a quadratic equation with coefficients depending on the other ten generators. The 
explicit (but very complicated) form of the equation was found by Nakamoto I7|, 
over Z, with respect to a slightly different system of generators. Revoy ^20 studied 
the field of rational invariants of two 3x3 matrices and also found eleven generators, 
with an explicitly given relation between them. 

In this paper we have found another natural set of eleven generators of the 
algebra C32 and have given the defining relation with respect to this set. It has 
turned out that our relation is much simpler than that in [17| . 

The first change in the set of generators is well known. In the traces of prod- 
ucts of degree > 2 we have replaced the generic matrices X, Y with generic trace- 
less matrices x,y, so our generators become tr(X),tr(F) and tr(x^), tr(xy), tr(?/^), 
tr(a;^), tr(x^y), tr(xy^), tr(y^), tr{x'^y'^),tr{x'^y'^xy). Extending diagonally the nat- 
ural action of GL2 on K ■ X + K ■ Y to C32, we have used the result of Abeasis 
and Pittaluga 1 and we have replaced the generators tr{x'^y'^), trix'^y'^xy) with ele- 
ments which generate one-dimensional GL2-inodules. It becomes clear that the only 
defining relation between our generators spans a one-dimensional G'i2-submodule 
of the subalgebra of C32 generated by all traces of products of x and y. Using 
several rules for computing with tensor products and symmetric algebras of irre- 
ducible GL2-modules, we have seen that the possible candidate for the relation is 
a linear combination with unknown coefficients of nine explicitly found elements 
in C32. In order to determine these coefficients, we have evaluated the hypothetic 
relation for various concrete traceless matrices x, y. The evaluations are equal to 0, 
and we have obtained a linear system with respect to the nine unknown coefficients 
of the considered linear combination. Using standard procedures of Maple, we have 
obtained the only solution of the system. Then, having the hypothetic defining 
relation, it is easy to check that it is really a relation. 

The paper is organized as follows. In Section 1 we give our system of generators 
and the defining relation between them. Our proof is direct and does not use 
any representation theory of GL2- In Section 2 we show how have we found our 
generators and the relation. We believe that our methods can be successfully used 
for other similar problems. 

The main results of the present paper have been announced in 0. 



It is a standard trick in the study of matrix invariants to replace the generic 
matrices in the traces of products with generic traceless matrices. We express X 
and Y in the form 



where e is the identity matrix and x, y are generic traceless matrices. Changing 
the variables xu and ya, we may assume that 



1. Main Results 



(3) 
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By well known arguments, as for "ordinary" generic matrices, without loss of gen- 
erality we may assume that a; is a diagonal matrix, i.e., 

/ 

(5) x = I X2 

V -ixi+X2) 

Till the end of the paper we fix the notation x,y for the two generic traceless 
matrices. Replacing X,Y with their expressions from (O, we obtain that, instead 
of the generators 1^ of Tcranishi, the algebra C32 is generated by the system 

tr(X),tr(y),tr(a:2),tr(a:y),tr(y2), 

(6) 

tr(a;'^), tr{x^y), tr(xy^), tr(y^), tr(x'^y^), tr(x^t/^xy). 

Now we replace the traces tr(x^y^), tr(a;^?/^a;?;) with elements more convenient for 
our purposes. We define 

(7) V = tr(x^y2) - tr{xyxy), 

(8) w — tr{x'^y'^xy) — tT{y'^x'^yx). 

The following lemma justifies the introduction of v and w. It expresses tr(x^j/^) 
and tr(x^y^xy) in terms of v,w and the other generators. 

Lemma 1.1. The following equations hold in C32.' 

(9) tr(x2y2) = + itr(a;2)tr(y2) + hr\xy), 

tr{x'^y^xy) — + ^tr{xy)v + j^tr(a;^)tr(a;2;)tr(?;^) 

(10) 

+ itr^(a:y) - itr(a;3)tr(y3) + ^tT{x'^y)tr{xy'^). 

Proof. For the proof of the first equality we shall make use of the Cayley-Hamilton 
theorem. For any 3x3 matrix z with eigenvalues zi, Z2, Z3 

z^ — eiz^ + e2Z — 636 = 0, 

where ei is the «-th elementary symmetric function in 21,22,23. The Newton for- 
mulas 



Pk - eipk-1 + ■■■ + {-If eu-ipi + {-Ifkek = 0, A: = 1, 2, 3, 

k. 
3 ■ 



allow to express the 's in terms of the power sums pk = z\ + + z^ ■ 



ei=pi, e2 = ^(pi-p2), 63 = i(2p3 - 3piP2 

Since pk — tr(z'^), if tr(z) = 0, then we obtain 

z^ ~ itr(z^)z - -tr(z^)e = 0. 

Multiplying by z and taking the trace we derive the trace identity for 3x3 traceless 
matrices 

(11) tr(z4) _ Itr^z') = 0. 
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Since the trace of the matrix x + y is equal to 0, the identity l(TT|l gives 
tr((x + y)4)-itr2((x + yn = 0. 

The multihomogeneous components of every trace identity are also trace identities. 
Considering the component of second degree with respect to x, and taking into 
account the invariance of the trace under cyclic permutation of the variables, we 
obtain 

4tr(x^y2) + 2tT{xyxy) - tr{x^)tr{y'^) - 2tr'^{xy) = 0. 

Replacing tr{xyxy) with tr{x'^y'^) — v from (O, we derive jnj- In order to prove 
we may proceed in a similar way. Instead, using Maple, we have evaluated 
(|10|l on the matrices x from Q and y from Q and have obtained that both sides 
are equal. □ 



(12) 



Now we define the following elements of C32 : 

tr(a;^) tr{xy) 



tT{xy) tT{y^ 



(13) 



(14) 



W5 



tr(a;^) tr(xy) tr(?/^) 
tr(a;'^) tr(a;^y) tr{xy'^) 
tr{x^y) ir{xy'^) tr(?/'^) 



(15) 



ii{x^) iT:{xy'^) 
ti{x'^y) tr(?/^) 



tr(?/3) tr(a;?;2) 
tr(a;2/^) irix^y) 



tr(a;^) tr(a;^y) 
tr(a;^2/) tr(a;2/^) 



(16) 



tr(a;^) tr(x?/) tr(?/^) 
tr(a;'^) tr(x^y) tr(a;?/^) 
tr{x'^y) tr(xj/^) tr(?/^) 



(17) 



w'^ = 5[tr3(y2)tr2(x3) + tr3(a;2)tr2(2/3)] 
— 30[tr2(y2)tr(a;j/)tr(x2y)tr(x'^) + tr2(a;2)tr(xy)tr(y^)tr(a;j/2)] 
+3{[4tr(y2)tr2(xy) + iT:'^{y'^)iT:{x'^)][?,tr'^{x'^y) + 2tr(xy2)tr(a;3)] 
+ [4tr2(xy)tr(x2) + tr2(a;2)tr(y2)] [3tr2(xy2) + 2tr(a;2y)tr(y3)]} 

-2[2tr3(a;?;) + 'itT:{x'^)tv{xy)tv{y'^)][^tr{xy'^)iY{x'^y) + tv{x'^)iv{y^)], 

where v is defined in |7J . The element w'-l can be expressed in the following simple 
way. Recall that a linear mapping 5 of an algebra i? is a derivation if 5{rs) = 
5{r)s + r5{s) for all r,s £ R. Every mapping 6q : {X,Y} C32 can be extended 
to a derivation 6 of C32 which commutes with the trace. In particular, defining 
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S{X) ^ 0, S{Y) — X, we obtain a derivation 5 which sends x to and y to x. Then 
(18) w'i = ^ j2{~iy5\ir^{y^))5'~\ir\y^)). 

i=Q 

The foUowing theorem is the main rcsuh of our paper. 

Theorem 1.2. The algebra of invariants C32 of two 3x3 matrices has the following 
presentation. It is generated by 

tr(X),tr(y),tr(a:2),tr(xy),tr(2/2), 

(19) 

tr(x'^), tr(a;^i/), tr(xy^), tr(y'^), v, w 
subject to the defining relation 

(20) ~ (^^1 - + + + ~ - - ^wr^ ^ 0, 

where the elements v,w,Wi,W2,w'^,w'^,Wi,W5,We,W'/ are given in 0, ||HJ, (|13|1 . 
Id, (Uni, ^ and |T7|). 

Proof. Replacing the system of generators with the system ^ , Lemma 11.11 
insures that the elements H19|l are generators of the algebra C32 . The relation l|20|l 
is checked with the help of Maple, where x is as in © and y is as in Q. 

Let us consider the polynomial algebra Pn in the 11 variables 1)19(1 written in 
the form Pn ^ (S'[tr(X), tr(r)])[w], where 

(21) S = K[tr{x^),tiixy),tiiy^),tiix^),tiix^y),tTixy^),tT{y''),v], 

and the principal ideal / = (/) of Pn generated by the element from (|20|l 

2 f I 2 4,1„1 2 1 4\ 

f = w ^ ^-w, - -W2 + -w, + -w, + -w, - -w, - -we - -u,. j . 

The factor algebra Pn/I is graded and is a free S'[tr(X), tr(y)]-module, freely 
generated by 1 and w. Therefore, the Hilbert series H{ti,t2) of Pn/I is 

(1 - t^){l - t2)[l - t?)(l - trt2){l - tl){l - t%){l - t^tl){l - tl){l - tjtj) ' 

and is equal to the Hilbert series Q of the algebra C32. Since the algebra C32 
satisfies the relation f — from (|20l) . it is a homomorphic image of Pn/I and 
the coefficients of its Hilbert series are bounded from above by the coefficients of 
the Hilbert series of Pn/I- Since the Hilbert series of C32 and Pn/I coincide, the 
algebras C32 and Pn/I are isomorphic and this completes the proof. □ 

2. How Did We Find The Relation? 

We refer e.g., to for a background on representation theory of GL^l and to 
for its applications in the spirit of the problems considered here. The natural 
action of the general linear group GLd on the vector space with basis {xi, . . . , Xd\ 
(or {Xi, . . . ,Xd}) can be extended diagonally on the free algebra K{xi, . . . ,Xd) 
and on the algebra generated by all formal traces 

K[tr{X,, ■■■X.Jlij ^l,...,d, m = 1, 2 . . .]. 
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This induces also a GLd-action on the algebra Cnd- It is well known that Cnd is 
a direct sum of irreducible polynomial GLd-modules. We denote by Wd{\) the 
irreducible GLd-module indexed by the partition A = (Ai, . . . , A^). We assume that 
Wd{X) = if A is a partition in more than d parts. 

Abeasis and Pittaluga P suggested the following way to describe the generators 
of Cnd- The subalgebra Ck of Cnd generated by all traces tr{Xi^ ■ ■ ■ Xi^) of degree 
m < k is a GLrf-submodule of Cnd- Let G^^^ be the homogeneous component 
of degree fc + 1 of Cnd- Then the intersection Ck H C^^^^^ has a complement 
Gfc+i in cll'j^^\ which is the Gi^-module of the "new" generators of degree k + 1. 
Without loss of generality we may assume that Gk+i is a submodule of the GLd- 
module spanned by all traces tr^Xi-^ ■ ■ •Xj^.^J. Abeasis and Pittaluga performed 
the calculations in the case of invariants of 3 x 3 matrices and obtained the following 
results for the GLd-module G = Gi©G2®- • • of the generators of C^d (the partitions 
in pP are given in "Francophone" way, i.e., transposed to ours): 

G = Wdil) © Wd{2) © WdiS) (B Wdil^) © Wdi2^) © VKd(2, 1^) 

(BWdiS, 1^) © Wdi2^, 1) © Wdil^) © WdiS^) © Wd{3, 1^). 

For d — 2 we may assume that Xi = X, X2 — Y. Then the algebra G32 has a 
system of generators which spans a Gi2-niodule 

M^2(l) © 1^2(2) © W2{3) © W^2(2^) © VF2(3^). 

The module W2(l) is spanned by the traces tr(X) and tr(y). Hence the subalgebra 
of G32 generated by traces of products of the traceless matrices x and y has a system 
of generators forming the module 

W^2(2) © W2{3) © M^2(2^) © W2{3^)- 

Let Ukd be the Gi^-module spanned by all formal traces tr{Xi^ ■ - • A".;^). The 
structure of Ukd is determined by the structure of the ^fc-module tr(Vfe) of the 
multilinear elements of Ukd, where Sk is the symmetric group of degree k and 
tr(Vfc) has a basis {tr{XkX„^i^ ■ ■ ■ X^(^k-i)) I cr £ Sk~i}- The 5'fc-module tr(14) is 
isomorphic to the induced module of the trivial one-dimensional module of a cycle 
of length k in Sk- For our purposes, in the following lemma we give a direct proof 
of the decomposition of Uk2 for fc = 2, 3, 4, 6. 

Lemma 2.1. The GL2-modules Uk2, k = 2,3,4,6, have the following decomposi- 
tions: 

U22 = W2{2), U32^W2{3), t/42 = W^2(4)©W^2(22), 

Ue2 = W2{6) © 2W^2(4, 2) © W2{3^). 

Proof. If a polynomial GL^-module Wd has the decomposition 

Wd = ^miX)Wd{\), m(A)>0, 

as a direct sum of irreducible components, then it has a natural grading and its 
Hilbert series is 

H{Wd, ti,...,td)=Y^ m{X)Sxiti, . . . , td), 

where S\{ti, . . . ,td) is the Schur function associated with A. The Hilbert series 
plays the role of the character of Wd and the multiplicities m(A) can be determined 
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in a unique way from the Hilbert series. For d = 2 the Schur functions have the 
following simple expression: 

The vector spaces Uk2, k — 2,3, 4, 6, have bases, respectively, 

{tr(X2),tr(Xy),tr(y2)}, 

MX^),triX^Y), tr{XY^), tr{Y^)}, 
{tT{X^), tT{X^Y), tT{X^Y'^), tT{XYXY), tT{XY^), tr(y'*)}, 
{tT{X^), tr{X'^'Y),tT{X^Y'^), tT{X^YXY),tT{X^YX^Y), 
tT{X^Y^), tT{X^Y^XY),tT{X^YXY^), tT{XYXYXY), 
tT{X^Y^),tT{XY^XY),tT{XY^XY^),tT{XY^), {tr(X^)}, 
Hence the Hilbert series of Uk2 are 

H{U22,ti,t2) —t\+tit2+t\ = S(^2){tl,t2), 
HiU32,h,t2) =4+ 1% + titl +tl = 5(3)(ti, ta), 

H{Ui2MM) =t{ + tlh + 2t\tl + htl + 4 

= S^4-){ti,t2) + S(^2^-)(ti,t2), 

H{Ue2, ti,t2) ^4+ t\t2 + 3t\tl + At\tl + 3ii4 + ^4 + 4 

= ^(g) + 2S'(4 2) + '5'(32)- 

This gives the desired decompositions. □ 

Lemma 2.2. The GL2-suhmodules W2(2^) C [/42 and W2(3^) C Uq2 are one- 
dimensional and are spanned, respectively, by 

V = v(X,Y) = tr(X2r2) - iT(XYXY), 

W = w{X,Y) = tiiX'^Y'^XY) - It{Y'^X'^YX). 

Proof. The Schur function 5(^2) (^1,^2) = (^1^2)'' gives that the module W2{h'^) is 
one-dimensional. The module Wd{X) is generated by a unique, up to a multiplicative 
constant, homogeneous element wx of degree \i with respect to Xi, called the 
highest weight vector of Wd(A). It is characterized by the following property, which 
follows from the results of De Concini, Eisenbud and Procesi and Almkvist, Dicks 
and Formanek see also Koshlukov ^1] for the version which we need. We state 
it for two variables and for Uk2 only. If wxlXjY) G Uk2 is homogeneous of degree 
(Ai, A2) and wxlXlY, Z) is the partial linearization of wx in Y (i.e., the homogeneous 
component of degree 1 in Z of wx{X, Y + Z)), then wx{X, Y) is a highest weight 
vector for some M^2(Ai, A2) if and only if wx{X\Y, X) = 0. Equivalently, if A is the 
derivation of the algebra generated by formal traces of products of X and Y , and 
A is defined by A(X) = 0, A(r) = X, then wx{X\Y, X) = A(u;a(^, Y)) = 0. Now 
the proof of the lemma follows from the equalities A(F) = A(W^) — which can 
be verified by direct calculations. For example, the partial linearization of V is 

v{X\Y, Z) = tT{X^YZ) + ti{X'^ZY) - 2iv(XYXZ), 

which gives that 

A(l/) = v{X\Y,X) = (1 + 1 - 2)ii{X^Y) = 0. 

□ 
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In order to find the expressions for V, W one may proceed in the following 
way, as shown for 1^2(3^). The generator W of W2(3^) is homogeneous of degree 
(3,3). Hence it is a linear combination of tT{X^Y^), tT{X^Y^XY), tr{Y^X^YX), 
tr{XYXYXY) and has the form 

W = r]itT{X'^Y^) + rj2tT{X^Y^XY) + ristiiY^X^YX) + r]itT{XYXYXY) 

with unknown coefficients ?7i , ^72 , % , ?74 ■ The partitial linearizations of the sum- 
mands are: 

for tr{X^Y^) : triX^XY"^) + tr{X^YXY) + tr{X^YYX) 
= 2tT{X'^Y^) + tT{X^YXY); 
for triX^Y^XY) : tT{X^XYXY) + tr{X^YXXY) + tr{X^YYXX) 

= triX'^Y^) + tr{X^YXY) + tr{X^YX^Y); 
for tr{Y^X^YX) : tr{XYX^YX) + tT{YXX^YX) + tr{Y^X^XX) 
= triX'^Y^) + tr{X^YXY) + triX^YX^Y); 
for tr{XYXYXY) : 3tT{X^YXY). 
The condition W{X\Y,X) = gives 

(2r/i + ?72 + m)i^{x^Y^) + im +m + m + 3r/4)tr(x3yxr) 

+ (?72 + ?73)tr(X2yx2r) = 0. 
Since the coefficients of tT:{X^Y'^),iT:{X^Y XY),iT:{X^y X'^Y) are equal to 0, we 
obtain the system 

2»yi+ V2+V3 =0 

V1+ '72+f?3+3??4 =0 

112 + m = 0. 

Its only solution is rji — r]4 — 0, 773 = —r]2- Our clement W is obtained for r]2 ^ 1. 

In this way we obtain that the subalgebra of C32 generated by products of 
traceless matrices is generated by the GL2-inodules W^2 (2), 14^2(3) spanned respec- 
tively by {tr(x^), tr(xj/), tr(2/^)} and {tr(a;^), tr(a:^y), tr(xj/^), tr(y^)}, and the GL2- 
modules 1^2(2^) and 14^2(3^) spanned, respectively, by v and w from and 0. 
Translating the resuhs of Teranishi [221, t^ie submodules ^^2(2), W^2(3) and ^2(2^) 
of C32 generate the subalgebra S from H21|l and the whole subalgebra generated by 
traces of products of traceless matrices is a free 5'- module with basis {l,w}. Since 
w'^ G S (B wS, we derive that w satisfies a relation of the form 

(22) f ^w^ +aw + b = 0, 

for some a, 6 £ S*. 

Lemma 2.3. The GL2-modules generated by the coefficients a and b in H22|l are 

isomorphic to W^2(3^) and ^2(6^), respectively. 

Proof. The elements g G GL2 act on w £ W^2(3^) by 

g{w{x,y)) = w{g{x),g{y)) = det^(.9) • w{x,y). 

Let us generate a GL2-module hy f — w'^ +aw + b from (|22|) . Acting with g G GL2, 
we obtain 

g{f) = detS(5) • + dct^g) ■ g{a)w + gib). 
Hence we have also the relation 

g{f) - dei\g) ■ f = det=^(5)(ff(a) - det=^(g) ■ a)w + {g{b) - det'ig) ■b)=0. 
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Since 1 and w form a basis of the free iS-module S ® wS, we obtain that g(a) — 
det'^lg) ■ a and g{b) = det^(f;) • b for aU g G GL2- Hence a and b generate GL2- 
modules isomorphic, respectively, to W2{3'^) and W2(6^). □ 

Lemma 2.4. In the decomposition of S as a GL2-module 

5 = 0to(Ai,A2)W^2(Ai,A2) 

we have to(3^) = and m(6^) = 8. 

Proof. The multiphcities m(Ai, A2) in the decomposition of S can be obtained using 
methods of the recent papers TUI . (In [S] Drensky and Genov calculated the 
generating function of the multiplicities for the whole algebra C32 and in Ji)_ they 
gave an easier way to calculate the multiplicities also for other objects similar to 
C32.) The multiplicities m(3^) and to (6^) can also be obtained directly, using well 
known rules of representation theory of symmetric and general linear groups. The 
first possibility is to find the coefficients of degree 6 and 12 in the Hilbert series of S 
and to express them in terms of Schur functions, as in Lemma l2.1l We shall give one 
more possibility. The algebra S is the tensor product of three polynomial algebras, 
namely the symmetric algebras i4r[T4^2(2)] = K[ti{x'^),ti{xy),tr{y'^)], K[W2{3)] = 
i4:[tr(a;^),tr(a;^?;),tr(a;y^),tr(?;^)] and K[W2i2'^)] = K[v]. The decomposition 

^[M^2(2)] = 0iy2(2Ai,2A2) 
is well known and follows from the equality 

n l^tt - ^ ^ 'S'(2Ai,...,2Ad)ftl7 ■ ■ ■,td)- 

l<i<j<d ' ^ 

The decomposition 

K[W2i2^)]=^W2i2b^) 

b>0 

is also well known. Using the multiplicities in the decomposition of ivr[W2(3)] given 
in [S] or calculating the first coefficients of the Hilbert series 

where hk is the homogeneous component of degree fc, or simply counting the basis 
elements of degree < 12 in iir[W2(3)], we obtain 

h^{tl,t2) = 5(3)(ti,i2), 
^6(^1, ^2) = S(Q){ti,t2) + 5'(4,2)(il, ^2), 
^12(^1, ^2) = '5(12) (^1, ^2) + '5(10,2) (^1 : ^2) + 'S'(9,3)(tl, ^2) + '5'(8,4)(^1: ^2) + '5(62) (i 1 , ^2 ) • 

This gives the parts of the decomposition of iir[W2(3)] which we need. Now, the 
homogeneous component of degree 6 of is obtained in the following way: 

^(6) = (i^[W^2(2)])(«) ® (X[W2(2)])(2) ® (if[M/2(22)])(4) © (X[W2 (3)] 

Using the Littlewood-Richardson rule, which in the case of GL2 states 

c 

W2{a + b,b) (g) W2{c + d,d) = ^W2{a + b + d + p,b + d + c - p), a > c, 

p=0 
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we calculate 

5(6) = (VK2(6) © 1^2(4, 2)) © 1^2(2) ® W^2(2') © {W2{6) © M^2(4, 2)) 

= 2t^2(6)©3VF2(4,2), 

and this means that m(3^) — 0. Similarly, we apply the Littlewood- Richardson 
rule to 

5(12) ^ (X [1^2 (2)]) (12) © {K[W2{2)]Y^'^ (i^[W^2(22)])(4) 
©(K[W^2(2)])(6) ® (X[iy2(3)])(^) © (if[M/2(2)])(^) © (if[M/2(22)])W 
©(if[M/2(2)])(2) ® (X[iy2(3)])(^) © (if[M/2(22)])(4) 
©(if[T4^2(3)])(i2) (if[T4^2(22)])(i2) 

and obtain the multiplicity m(6'^). We shall mention that W2(62) participates with 
multiplicity one in the components 

(X[Ty2(2)])(i2)^ (^[y^2(2)])(8) ® (X[Ty2(22)])(4), (i^[W2(2)])W (i;f[Ty2(22)])(8), 

(if[W2(2)])(2) ® (if [iy2(3)])(6) ® iK[W2i2^)])^^\iK[W2m)^'^\iK[W2i2^)])^'^^ 
and with multiplicity 2 in the component {K[W2{2)])'^'^'> © (if [T4^2(3)])(^' . □ 

Lemma 2.5. T/ie elements wi, W2, W3, u'3 , W4, ^5, we, W7 (^iwen m p3|l . (|14|l . p5|l . 

(|16|l and (|17|l are highest weight vectors of submodules ^2(62) and are linearly 
independent in S . 

Proof. It is a direct checking to see that the eight elements are linearly independent. 
All these elements are of degree (6,6). In order to see that they are highest weight 
vectors it is sufficient to verify that each Wi(x,y) has the property Wi{x\y,x) = 0, 
as in the proof of Lemma □ 

We think that, for possible applications, it is important to know how we have 
found the elements wi , u'2 , W3 , w'^ , W4,W5,wq,wj. The element v from Q belongs 
to W2(22) and is a highest weight vector. Similarly, u from H12|l is a highest weight 
vector in W2{2'^) C (if [W2(2)])(''^ This explains why wi,W2,W4 and wr are high- 
est weight vectors in the submodules 1^2(62) of {K[W2{2)]Y^^\ (is:[iy2(2)])(^) © 
(if[Ty2(22)])(4), (if[Ty2(2)])(4) ©(i4:[W2(22)])(8) and (if [M^2(22)])(i2), respectively. 
The tensor product (if [1^2 (2)]) (2) © (if [T4^2(3)])(^) © (if [iy2(22)])(4) contains one 
submodule W2(62). Since (if [T4^2(22)])('i' = W2{2'^) is one-dimensional and spanned 
by the highest weight vector v, we need to find a highest weight vector of the sub- 
module 1^2(42) of (if [Ty2(2)])(2) © (if [11^2(3)] )(^). It has been done by the method 
of Lemma 12.21 We are looking for a linear combination i(j(42-)(x, of products 
tr(ziZ2)tr(z3Z4Z5)tr(z62;72;8), Zi = x,y, which is homogeneous of degree (4,4) and 
satisfies the condition W(^42-^{x\y,x) = 0. All possible summands are 

tr(a;2)tr(a;2?;)tr(y^), tr(a;2)tr2(a;y2)^ 

tr(a;y)tr(a;'^)tr(y"^), tr(a;y)tr(a;2j/)tr(a;y2)^ 

tr(y2)tr(a;^)tr(a;?/2),tr(2/2)tr2(a::2y). 

We determine the coefficients of these summands from the condition W(^42-^{x\y, x) = 
0. It has turned out that 1/7(42) has a determinantal form and in this way we obtain 
W5. The highest weight vector wq S (if [W2(3)])'-12) is obtained similarly as a 
linear combination of homogeneous products of four ir[zi^Zi.^Zi^), of degree (6,6), 
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Zi = x^y. Finally, the proof of Lemma f2.4l gives that the multiplicity of ^2(6^) in 
(K[W2(2)])(6) {K[W2{2.)])'~^^ is equal to 2. Since 

{K[W2{2)]f^ - (i^[W^2(3)])(^' = W^2(6)©W^2(4,2), 

the Littlewood-Richardson rule gives that we may fix one of the highest weight 
vectors of the two copies of ^2(6^), namely Wg, in W^2(4, 2) (g) ^2(4, 2), and the 
second, namely w'^, in ^^2(6) ® ^^2(6). For Wg we use the same arguments as for 
W5, replacing v with u G (/'ir[VF2(2)])^''-'. For Wg, from the expression (|18|1 and the 
equalities 5"^ {tv^ [y"^)) = 5'^ {tv'^ {y'^)) = we obtain w'^{x\y,x) = 6{w3{x,y)) = 0. 

Lemma 2.6. There exist constants ^1,^2, £.3,^3 TS.iT ^5, £,6, S.7 such that the relation 

(23) - {£iwi + £,2W2 + Ca^^s + + + ^5^5 + ^e^e + ^wy) = 
holds in 032- 

Proof. By Lemma [2.31 the coefficients a and b in (|22|l belong to GL2-submodules 
of the algebra 5* isomorphic to W2{3^) and ^2(6^), respectively. Lemma IT^ gives 
that 5* does not contain a submodule W2(3^). Hence the coefficient a is equal to 
and w satisfies a relation + 6 = for some b G ^^2(6^) C S. 

Representation theory of GL^ gives that if VFi, i = 1, . . . ,k, are k isomorphic 
copies of Wd{X) and Wi G Wi are highest weight vectors, then the highest weight 
vector of any submodule WdiX) of the direct sum Wi ® • • • © Wk has the form 
^iWi + • • • + SkWk for some G K. Any /c linearly independent highest weight 
vectors can serve as a basis of the GLd-module Wi ® • ■ • © Wk- By Lemma IT^ 
W^2(6^) participates in S with multiplicity 8 and Lemma |2 . 51 gives such 8 linearly 
independent highest weight vectors. Hence w satisfies the desired relation. □ 

Now the problem is to find the coefhcients ^1,^2,^3,^31^4:1^57^61^7 i'^ the rela- 
tion 1)23(1 . We have proceeded in the following way, using Maple for the concrete 
calculations. 

We have evaluated H23() for various traceless matrices. First, we assume that x 
is as in (0) and 

/ f 

J/= f 
\ f 

The only nonzero expressions are for w'^ and wq. Comparing the coefficients of 
Xi and x\x\, we obtain that 

4 - 360^3 = and - 3 - (2f 60^3 + 8f ^g) = 0, 

which gives that 

(24) ^3=1/90, ^6 = -l/3. 
The second step is to use 
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r 

y= 


f 




V 







V 







which gives w — Q and 27^i - 


-9?^ 


+ 36^^' 


- 3^6 = 0. 


Hence, by 



(25) 27^1 - 9^3 = -7/5. 
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Then, again we fix a: as in (pj and 

y = 

We obtain w = and 

(64^ + 166 - + 72^^' + 4^4 - 2^5 + ^7)xt 
+ (192a - 56^^ + 144^^' - 12^ - 8^5 - 12^6 - 6C7)xlx2 
+(3846 - 136^^ + 504^^' + 12^ + 2^ - 48^ + 15^7)xW2 
+(4486 - 326 - 176^3 + 864^3 - 86 + 166 - 726 - 206)a;?x3 
+ (384^1 - 136^3 + 504^3 + 126 + 26 - 486 + 156)a;?a;^ 
+ (1926 - 56$^ + 144^^' - 126 " 86 " 126 - Q^7)xixl 
+(646 + 166 - 8^3 + 72C^' + 46 - 26 + 6)2^2 = 0. 
Since the coefficients of x\x2~'' are equal to 0, this gives four more relations for the 
6s. Combining with 1241) and H25|l . we have 

6 - -1/54 + 1/26 + 3/46, 6 - 1/54 - 3/26 - 7/46, 

(26) 

6 - 1/10 + 3/26 + 9/46, 6 = -4/9 + 3/26. 
Finally, using x from |SJ and y from Q), together with the expressions for the 6s 
from (|2Sl and we annulate the coefficient of a;i2;2J/iiyi2?/232/3i to derive 

-276 - 81/26 + 3 = 0. 
Similarly, the coefficient of Xixlyi2y22y23y3iy32 gives 

366 + 906 + 4/3 = 0. 

Hence the only possibility is 

6 = 1/3. 6 = -4/27 
and we conclude that the relation is the given in (|20|l . 
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